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Abstract 

This  report  documents  the  development  and  use  of  three  dimen- 
sional graphics  display  programs  developed  in  conjunction  with  the 
simulation  of  muscle  action  and  blood  flow  in  the  heart.  A  series  of 
color  photographs  are  included  to  illustrate  its  capabilities.  Its  fea- 
tures are  described  and  details  of  implementation  are  discussed.  Some 
areas  of  future  research  are  outlined.  The  report  includes  a  short  user 
manual. 


1      Introduction 

A  mathematiccil  model  of  muscle  action  and  blood  dynamics  in  the  heart  and 
a  computational  method  for  solving  the  equations  of  this  model  are  described 
in  !l,2].  The  heart  is  modeled  as  a  collection  of  elastic  fibers  immersed  in 
blood,  a  viscous  incompressible  fluid.  Given  an  initicd  configuration  of  the 
liber  structure  and  fluid  markers,  the  computational  method  (HEART3D) 
outputs  the  updated  positions  of  fibers  and  markers,  as  well  as  velocity  and 
pressure  fields,  for  selected  time  steps  in  the  simulation.  It  was  important 
to  present  this  vast  arrav  of  numbers  to  the  user  in  a  meaningful  manner. 

It  is  widely  accepted  that  interactive  computer  graphics  is  a  powerful 
form  of  communication.  For  this  application,  graphics  is  used  to  enable 
visualization  of  flow  inside  an  object  in  notion.  The  Evans  &  Sutherland 
PS  340  Computer  Graphics  System  is  suited  for  this  purpose.  In  order  to 
use  the  PS  340,  programs  had  to  be  developed  to  tailor  the  use  of  this  device 
for  the  needs  of  the  HEART3D  program.  Two  important  properties  of  the 
simulation  guided  development:  it  is  three  dimensional  and  dynamic.  The 
PS  340  system  includes  a  high  speed  vector  graphics  display  which  allows 
the  user  to  interact  with  the  display  in  real-time;  its  raster  display  can  be 
used  to  model  solid  objects. 

This  report  documents  the  development  and  use  of  a  set  of  programs 
to  display  data  generated  by  HEART3D.  In  Section  2  we  discuss  the  design 
considerations  of  the  programs.  Section  3  contains  an  overview  of  the  PS  340 
system  and  the  capabilities  that  make  it  suitable  for  this  purpose.  The  data 
structures  used  by  HEART3D  for  output  are  described  in  Section  4.  Sec- 
tion 5  is  a  presentation  of  the  display  capabilties  of  the  programs  developed, 
while  Section  6  describes  the  implementation  of  the  programs.  Some  ideas 
for  future  development  are  discussed  in  Section  7.  Section  8  contains  a  sum- 
mary of  our  experiences  in  developing  these  programs.  Appendix  A  is  a  user 
manual  for  the  graphics  programs. 


2      Considerations 

As  stated  in  the  Introduction,  graphics  are  used  to  present  the  output  of 
the  HEART3D  program  to  allow  users  of  the  progrsan  to  visualize  fluid  flow 
within  an  object  in  motion.  Thus  a  collection  of  programs  had  to  be  designed 
to  interpret  the  output  of  HEART3D,  control  a  graphics  device  and  provide 
a  user  interface. 

Usually,  software  design  proceeds  from  a  set  of  formal  specifications. 
Even  numerical  software  is  specified  by  the  mathematical  models  to  be  im- 
plemented and  the  numerical  scheme  selected.  However,  there  are  a  number 
of  times  when  the  computer  is  to  be  used  as  an  experimental  tool,  preclud- 
ing the  formalization  of  such  specifications. In  the  case  of  developing  these 
graphics  programs,  it  was  not  possible  to  construct  a  set  of  specifications, 
however,  a  set  of  considerations  were  formed. 

This  application  had  two  general  design  criteria:  it  was  dynamic  and 
rhree-dimensional.  The  simulation  was  dynamic  in  the  sense  that  it  modeled 
phenomena  that  changed  over  time.  The  output  from  the  simulation  was  a 
series  of  files,  referred  to  as  frames,  that  showed  the  new  positions  of  the 
fibers,  fluid  markers  and  the  new  velocity  and  pressure  fields  from  points 
in  the  main  iteration  loop.  Thus,  the  graphics  system  had  to  present  these 
frames  in  sequence;  the  PS  340  system  can  present  this  sequence  fast  enough 
to  impart  a  sense  of  motion. 

The  three-dimensional  aspect  of  HEART3D  posed  other  challenges.  User 
interaction  with  the  graphics  display  became  much  more  important,  and 
there  was  a  continuous  effort  to  find  better  ways  to  present  three-dimensional 
information.  Our  current  efforts  are  aimed  at  presentation  of  three  dimen- 
sional vector  fields,  a  non-trivial  task  (see  Section  7.3). 

In  addition  to  these  two  problem  considerations,  the  software  was  to  be 
used  in  two  different  ways:  to  analyze  the  operation  of  HEART3D  and  to 
make  presentations  for  showing  to  others. 

Understanding  HEART3D  was  the  first  concern.  The  code  for  the  sim- 
ulation was  extremely  complex  and  was  tested  incrementaUv.that  is,  first 
the  code  was  tested  for  a  fiber-wound  torus,  then  for  a  fiber-wound  torus 
with  a  "'bulge",  and  so  on.  Some  method  of  presentation  was  needed  so  that 
one  could  gain  a  global  qualitative  understanding  of  how  the  simulation  was 
progressing.  This  then  led  to  other  programs  that  used  the  same  data  to 
make  more  quantitative  studies.  For  example,  when  peristaltic  motion  on 
the  fiber-wound  torus  was  being  tested,  the  graphics  display  was  used  to 
determine  if  the  fibers  behaved  properly  in  the  presence  of  a  moving  wave 
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of  contraction.  This  was  best  done  by  "'eye-balling"  the  experimental  re- 
sults, hypothesizing  about  the  behavior  and  then  modifying  the  code  and 
performing  further  tests.  In  this  sense,  the  development  of  HEART3D  fol- 
lowed a  cycle  similar  to  that  used  in  experimental  laboratories.  This  made 
the  development  of  the  display  programs  more  difficult  since  each  time  an 
experiment  was  run,  the  attention  was  focused  on  a  different  part  of  the 
simulation,  so  more  features  were  added  or  original  features  were  modified. 

The  second  concern  was  presentation  of  the  restilts  to  a  wider  audience. 
This  followed  a  cycle  akin  to  that  of  animated  motion  picture  creation:  put 
a  frame  on  the  screen,  take  a  picture  of  it.  advance  the  camera  and  repeat. 
The  issues  here  were  control  of  the  PS  340  display,  selection  of  video  and 
photographic  equipment  and  interfacing  automated  video  equipment  to  the 
display. 

These  guidelines  led  to  a  list  of  considerations  for  progrcim  development: 

L.  The  output  of  HE.\RT3D  is  to  be  organized  by  experunent,  where  an 
experiment  is  a  sequence  of  frames,  each  containing  fiber,  marker  and 
velocity  data  for  one  time  step. 

2.  The  output  should  be  displayed  as  an  animation.  The  number  of 
frames  displayed  per  second  should  be  large  enough  to  give  an  impres- 
sion of  motion  of  the  hbers  and  the  fluid. 

3.  The  user  must  be  able  to  interact  with  the  display.  This  should  include 
translation,  rotation  and  scaling  of  the  fiber  structure,  controlling  an- 
imation speed  and  alternately  displaying  and  hiding  portions  of  the 
simulation. 

4.  Permanent  records  on  film  or  video  tape  should  be  made.  Since  this 
is  time-consuming,  it  should  be  automated  as  much  as  possible. 

5.  The  design  of  the  graphics  prograjns  should  be  as  fle.xible  as  possible 
given  the  other  considerations. 


3      Evans  &:  Sutherland  PS  340  Computer  Graph- 
ics System:  An  Overview 

The  PS  340  graphics  system,  a  member  of  the  PS  300  family  of  graphics 
systems  from  Evans  &  Sutherland,  was  chosen  for  development  because 
it  satisfied  many  of  the  design  criteria  for  the  problem,  as  described  in 
Section  2.  The  salient  features  of  the  PS  340  are 

•  Control  of  three-dimensional  models.  The  "'primitive"  display  compo- 
nent of  the  PS  340  is  a  three  dimensional  object.  Thus,  it  was  unnec- 
essary to  explicitly  compute  the  transformations  for  three  dimensional 
objects  onto  a  two  dimensional  display  space;  this  was  done  implicitly 
by  the  system. 

•  Local  manipulation  of  models.  Once  initialized,  display  objects  cotild 
be  manipulated  (i.e.,  rotated,  translated,  scaled,  etc.)  entirely  by  the 
PS  340  system. 

•  Real-time  interaction.  The  PS  340  provides  a  number  of  user  control 
devices  (see  Section  3.1).  Programs  can  be  written  to  use  these  devices 
in  any  fashion  the  application  demands. 

•  Hierarchically  structured  models.  Display  objects  can  be  organized 
hierarchically,  with  each  subcomponent  under  user  control. 

•  Advanced  three  dimensional  visualization  of  solid  objects.  The  PS  340 
provides  special  firmware  capabilities  to  defijie  and  manipulate  solid 
objects.  Solid  objects  are  specified  as  a  collection  of  polygons.  Manip- 
ulations include  cross-sectioning  and  hidden-line  and  surface  removal. 
The  PS  340  raster  display  can  be  used  to  generate  smooth  shaded 
images  of  solid  objects  with  multiple  sources  of  illumination. 

.\  brief  overview  of  the  hardware  and  software  components  of  the  PS  340  are 
given.  The  reader  is  directed  to  31  for  a  complete  description  of  the  PS  340 
system. 

3.1      Hardware 

The  PS  340  svstem  is  a  dedicated  graphics  device  consisting  of  a  processing 
unit,  a  vector  graphics  display,  a  raster  graphics  chsplay  and  a  number  of 
input  devices.   Figure  1  is  a  schematic  diagram  of  the  system. 
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Fieure  1:  Schematic  diagram  of  the  PS  340  graphics  system.The  GCP.MM 


and  DP  are  functional  subunits  of  the  processing  unit.  There  are  two  modes 
of  communication  between  the  host  and  the  PS  340:  asynchronous  and 
ETHERNET. 


The  processing  unit  is  built  around  the  Motorola  68000  microprocessor 
and  has  three  components:  the  graphics  control  processor  (GCP),  display 
processor  (DP)  and  main  memory  (MM).  The  GCP  is  responsible  for  host 
communications,  terminal  emulation,  input/output,  and  interpreting  the 
PS  340  command  language  (described  in  3.2.  The  DP  traverses  the  list  of 
active  display  objects  and  drives  the  vector  and  raster  displays.  The  MM 
contains  user  programs,  display  objects  and  the  system  software. 

The  vector  display  is  a  device  that  is  optimized  for  drawing  wire-frame 
objects.  Drawing  a  line  between  two  points  on  the  screen  is  accomplished  by 
changing  the  magnetic  field  of  the  electron  gun  in  the  tube  so  that  the  beam 
sweeps  out  a  line  on  the  tube.  This  is  in  contrast  to  a  standard  television 
display  where  the  electron  beam  sweeps  out  a  fixed  pattern.  The  PS  340  can 
display  65,000  vectors  on  the  screen  at  one  time,  and  has  a  choice  of  around 
1.000  colors.  The  advantage  of  using  the  vector  display  is  that  it  facilitates 
real-time  interaction  with  the  objects  being  displayed.  The  disadvantage  is 
chat  it  cannot  be  interfaced  directly  with  a  video  cassette  recorder  (although 
a  video  camera  can  be  used  to  record  what  is  being  displayed  on  the  screen). 

The  raster  display  is  a  device  that  functions  much  like  a  standard  tele- 
vision set.  Points  on  the  display  are  individually  addressable  as  pixels;  the 
PS  340  system  has  a  resolution  of  640  by  480  pixels.  Each  pixel  can  be  one 
of  2^'*  colors.  The  system  has  firmware  that  optimizes  -ulid  object  modeling 
by  providing  features  such  as  hidden  line  and  surface  removal,  illumination 
and  shading  of  sohd  object  models.  It  can  also  be  interfaced  directly  with  a 
video  cassette  recorder. 

A  number  of  input  devices  are  included.  The  kevboard  has  a  standard 
layout;  it  is  primarily  used  by  the  terminal  emulation  software  so  that  the 
PS  340  can  act  Uke  a  terminal  when  communicating  with  the  host.  It  can 
also  be  used  by  appUcations  programs.  The  system  also  has  a  box  with  eight 
dials  for  use  by  applications  use.  Both  the  dials  and  the  function  keys  have 
LED  displays  above  them  so  that  the  user  may  label  the  dials  and  keys  as 
necessary.  Finally,  a  graphics  tablet  is  provided,  allowing  users  to  point  at 
objects  on  the  screen,  draw  new  objects,  etc. 

The  PS  340  is  designed  to  be  host  independent,  that  is.  it  can  be  in- 
terfaced to  ciny  size  or  make  of  host  computer.  It  communicates  with  the 
host  through  a  number  of  methods:  asynchronous,  parallel,  direct  memory 
access  (DMA),  and  ETHERNET  interfaces  are  implemented.  The  system 
can  emulate  a  VTIOO  terminal,  allowing  the  user  to  communicate  with  the 
host  in  a  natural  way. 


3.2      Software 

The  PS  300  family  of  graphics  systems  has  its  own  command  language  with 
two  equivalent  representations:  ASCII  and  binary.  The  ASCII  form  con- 
sists of  English-like  commands  which  correspond  to  typical  operations  in 
computer  graphics  applications.  The  VECTOR-LIST  command  creates  an 
object  as  a  list  of  vectors,  a  set  of  points  and  the  lines  that  connect  them. 
Commands  such  as  ROTATE  are  provided  to  manipulate  the  objects.  The 
WINDOW  command  (as  well  as  others)  control  representation  of  objects  on 
the  screen.  The  DISPLAY  command  produces  an  object  on  the  screen,  and 
so  on.  Using  these  commands,  the  user  builds  structures  which  represent 
the  objects  to  be  displayed.  Memory  is  treated  as  a  collection  of  objects, 
each  with  a  distinct  name,  so  users  are  relieved  from  the  burden  of  keeping 
track  of  addresses  in  memory.  Objects  are  groups  of  graphical  data,  matrix 
transformations  and  attribute  defLoitions.  Objects  can  be  used  as  templates 
to  compose  larger  objects.  Commands  in  the  language  fall  into  one  of  five 
categories: 

•  Data  Structuring.  These  are  used  for  creating  object  structures  and 
naming  and  refrencing  them.  Objects  cam  be  conditionally  referenced 
also.  Data  structuring  commands  form  display  trees  which  is  a  hi- 
erarchical grouping  of  graphical  data,  transformations  and  attributes 
which  define  cin  object. 

•  Modeling.  These  are  used  to  create  primitive  objects  such  as  vector 
lists,  curves  and  polygons.  Also,  three-dimensional  transformations 
such  as  scaling,  rotation  and  translation  fall  into  this  category. 

•  Viewing.  These  commands  create  different  views  of  objects,  such  as 
an  orthogonal  or  perspective  views. 

t  Attribute  Setting.  These  commands  set  and  change  aspects  of  a  model. 
Appearance  attributes  determine  aspects  such  as  color  and  intensity- 
Structure  attributes  allow  conditional  display  of  only  a  part  of  model's 
structure. 


• 


Immediate  Action.  These  are  used  to  display  and  remove  objects  from 
the  screen.  They  also  build  function  networks. 


X  key  idea  in  programming  the  PS  340  is  the  function  network.  This  is 
a  data-driven  approach  to  programming.  Each  function  is  a  transformation 


from  its  inputs  to  its  outputs;  functions  are  instantiated  (i.e.,  an  instance  of 
a  function  is  created  for  user)  from  a  collection  provided  in  the  Command 
Language.  A  function  instance  does  not  do  anything  until  data  is  presented 
to  its  inputs.  Many  commands  in  the  language  are  associated  with  func- 
tion nodes;  this  allows  networks  to  modify  structures  defined  by  commands. 
Function  networks  are  explained  in  i3]. 

There  are  three  different  methods  of  loading  applications  to  the  PS  340. 
The  user  can  type  the  program  directly  into  the  system  in  command  mode; 
this  is  best  for  small  applications  or  when  some  algorithm  is  being  tested. 
An  application  can  be  written  on  the  host  machine  using  available  text  ed- 
itors and  then  down-loaded  to  the  PS  340  using  cin  appropriate  command. 
This  method  is  best  for  larger  applications  that  are  subject  to  continuous 
modification.  Finally,  an  application  can  be  written  in  PASCAL  or  FOR- 
TRAN and  the  Graphic  Subroutines  provided  by  Evans  &  Sutherland  can 
be  used  to  interface  to  the  system.  These  subroutines  (referred  to  as  GSRs) 
convert  the  ASCII  commands  to  their  binary  format  equivalent.  L'sually, 
a  combination  of  these  methods  is  used.  In  our  application,  the  simulation 
data  was  down-loaded  by  a  PASCAL  program  but  the  user  interface  pro- 
gram was  written  in  the  command  language  and  down-loaded  as  an  ASCII 
text  file. 

3.3      Data  Flow 

Figiire  2  shows  the  tiow  of  information  between  the  various  systems  used 
in  the  simulation.  The  HEART3D  program  runs  on  a  CRAY  2  machine 
located  at  the  Minnesota  Supercomputer  Center.  Output  is  brought  to  the 
VAX/VMS  cluster  at  New  York  University  via  the  ARPANET  or  by  sending 
a  magnetic  tape.  The  PS  340  was  connected  to  the  VAX  over  a  9600  baud 
asynchronous  line.  In  addition,  the  PS  340  is  connected  to  a  Lyon-Lamb 
VAS  IV  video  controller.  NTSC  television  signals  are  input  to  this  controller 
from  the  raster  display  and  a  seriaJ  line  carried  control  signals  back  and  forth 
from  the  controller  to  the  PS  340.  The  controller  is  driven  indirectly  by  the 
VAX  by  Graphic  Subroutine  calls  to  the  PS  340.  Subroutines  were  written 
allowing  the  productionof  video  tapes  to  be  automated  (this  is  discussed  in 
Section  7.1). 
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Figiire  2:  Diagreim  of  information  flow  for  the  simulation.  The  boxes  are  the 
fimctional  imits;  the  hardware  and  software  used  is  given.  Each  data  path 
is  labeled  with  the  mechanism  used  to  transfer  data  back  and  forth.  Note 
that  the  VAS-LV  controller  is  managed  by  the  VAX/VMS  system  indirectly 
through  calls  to  the  PASCAL  Graphic  Subroutines  to  the  PS  340. 
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4      Data  Structures 

The  first  task  of  the  graphics  programs  is  to  process  the  collection  of  ASCII 
data  files  generated  by  HEART3D  during  the  simulation.  To  this  end,  the 
data  structures  used  by  both  programs  have  been  standardized.  These  struc- 
tures are  organized  hierarchically. 

The  topmost  level  of  organization  is  the  frame.  A  frame  is  the  collec- 
tion of  data  files  that  HEART3D  generates  after  k  iterations,  where  ^  is  a 
number  determined  before  execution  and  is  represented  in  the  data  files  as 
the  KLOK  value.  A  frame  contains  the  configuration  of  fibers,  positions  of 
fluid  markers,  and  velocity  and  pressure  fields.  Frames  are  identified  by  the 
dimensions  of  the  computational  domain,  the  experiment  number  and  the 
KLOK  value.  Each  frame  is  organized  so  that  it  may  be  displayed  alone  or 
as  part  of  aji  animation  sequence,  thus  a  number  of  data  fields  are  replicated 
in  each  file. 

The  muscular  structure  of  the  heart  is  modeled  as  a  collection  of  fibers. 
Each  fiber  is  represented  as  a  sequence  of  points;  in  the  wire-frame  display 
(Section  6,  fibers  are  presented  as  a  closed  path  of  line  segments  joining 
these  points.  Fibers  are  subdivided  into  groups;  these  may  be  selectively 
displayed,  as  illustrated  in  Section  5.  Currently,  there  is  a  maximum  of 
2500  points  in  ail  groups  of  ail  fibers  in  a  frame. 

Another  component  of  frame  data  is  the  fluid  markers.  This  data  struc- 
ture is  maintained  to  allow  the  user  to  visualize  the  fluid  flow  in  the  heart. 
Thev  are  ogranized  into  shells,  so  that  flow  in  different  regions  of  the  simula- 
tion to  be  studied.  Each  marker  is  represented  by  its  Cartesian  coordinates. 
Currently,  each  frame  contains  the  e.y.z  positions  of  5600  fluid  markers. 

Both  velocity  and  pressure  field  data  are  organized  on  a  lattice  encom- 
passing the  domain  of  computation.  Velocity  are  three  dimensional  vectors, 
while  pressure  is  a  one- dimensional  quantity. 
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Figure  3:  Markers  in  initial  configuration. 

5      Display  Capabilities 

In  this  section,  the  capabilities  of  the  programs  are  presented  using  a  se- 
quence of  photographs.  The  dvnanaic  nature  of  the  program  must  be  em- 
phasized: these  photographs  were  taken  usmg  the  dynarmc  and  raster  dispiav 
programs  documented  in  Section  6  without  modification.  The  photographs 
are  taken  from  a  simulation  of  peristaltic  motion  in  a  fiber- wound  torus  as 
discussed  in  '21. 

Figure  3  shows  the  fluid  markers  in  the  initial  configuration,  that  is. 
at  time  step  zero.  As  described  in  Section  4.  markers  are  orgamzed  into 
cvlindrical  shells  within  the  fiber  structure  (wfiich  is  not  shown).  The  lower 
left-hand  corner  indicates  the  data  files  being  displayed  bv  showing  the  di- 
mension, experiment  number  and  KLOK  value  of  the  current  frame.  The 
upper  left-hand  area  of  the  screen  displays  information  about  the  view  such 
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Figure  4:  Fluid  markers  at  KLOK  =  768. 
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Figure  5:  Even/odd  display  of  markers  in  initial  configuration. 

as  the  x.y.z  translation  values:  the  number  of  degrees  chat  the  object  has 
been  rotated  about  the  x.y  and  z  axes:  the  scale  factor  applied  to  the  ob- 
ject and  the  number  of  frames  to  be  displayed  per  second.  Figure  4  shows 
the  same  view  of  the  markers  for  a  later  step  in  the  simulation,  namelv  for 
KLOK  =  768. 

In  figure  5.  the  same  configuration  of  markers  is  shown  as  in  figure  3  but 
selected  portions  of  the  markers  have  been  "turned  off*'.  This  allows  the 
user  to  watch  how  particular  markers  move  as  the  simulation  progresses. 
The  markers  are  divided  into  "even"  and  "odd"  groups:  each  group  may 
be  seiectivelv  turned  on  or  otf.  One  important  feature  of  the  user  interlace 
is  evident  here:  that  of  orthogonal  design.  This  refers  to  the  fact  that 
each  capabilitv  of  the  display  programs  is  usable  independent  of  the  other 
'•apabilities.  This  allows  the  user  to  concentrate  on  particular  aspects  ot  the 
display.  Figure  6  shows  the  same  view  at  KLOK  -  768.  Note  that  it  is  now 
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Figure  6:  Even/odd  display  for  KLOK  =  768. 


15 


Ftgtire  T:  Stereo  abplay  of  fibers  utA  soinE  ttmthits  m  th^.  mitial  conHgura- 

tio.B, 

posiibk  to  sec  tliat  itiafk^r?  have  moved  arDuiid  the  tOf^s  since  one  c»%3s  sti.!]. 
identify  tiic  ■st'paratft  grovpmgs. 

The  next  four  pkotograplis  fkmonstfate  the  slered'  tiipabiUty  of  the  dy- 
iiatjiic  display  program,  'Jhl$  h  described  in  Secdon  $.4.2.  The  stereo  effecS: 
is  created  bv  placing  s  red  tdtti'f  ov€f  the  fight  eye  aad  a  blue  ftllCT  ove? 
the  kfl  eye  before  viewing  the  pktiti«$,  IiKXperMlve  red/blue  glasses  ca?i 
be  oblftined  Irora  naveity  shops  chat  aJlovv  vkwing  the  stereo  effect  hi  th^se 
phoiograpbs.  Figutft  7  i^hows  botii  the  fib«r  strutturt^  and  fluid  markeis  fo? 
the  torus  us<'d  in  HEAKT3D.  The  "odd'"  markers  !mv«  heeris  turi^ed  oC  as 
well  ss  several  mii«T  shells  id  Jiisfters  to  prevent  the  display  ftom  being 
cluttered  up.  Figure  3  •shvvvs  the  game  view  sbr  KLOK  -  768,  The  ssext  two 
paotograpljs  (Hgures  9  and  IS  aiso  show  these  views  rotated  abwut  botlis  the 
X.  and  V  axes  30  degrees. 
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Figure  S;  Ss^ereo  dispiay  ai  KLQK  -  Ti5S, 
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Figure  9;  Rotated  il<?reij  'ijsfilay.  ThU  is  the  saiiie  dat&  m  above.  Ihe  iuruN 
htis  btta  fotat^d  about  tlse  x  aiid  y  axes  ^JO  di'^rees. 


IS 


Figure  10;  RMfated  stereo  display  for  KLOK  —  7i>S. 
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Figure  11:  Rasfer  display  r4' corus 


20 


Figtire  11  shows  the  results  of  preliminary  experiments  with  the  PS  340 
raster  display.  The  fiber  structure  is  modeled  as  a  sohd  object  wrapped  with 
ribbons.  This  work  is  further  described  in  Section  7.1.  The  PS  340  allows 
the  user  to  establish  multiple  sources  of  illumination  and  control  the  display 
attributes  of  the  object  such  as  its  color  and  specular  and  diffuse  reflective 
qucdities.  For  further  details  on  the  theory  of  solid  object  modeling,  the 
reader  is  directed  to   4l. 
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6      The  Dynamic  Display  Program 

The  programs  developed  used  the  PS  340  vector  display  and  input  devices  for 
a  dynamic  wire-frame  depiction  of  the  simulation.  A  sequence  of  frames  were 
down-loaded  to  the  PS  340  from  the  VAX.  A  clock  function  node  was  used  to 
select  the  current  frame  to  display,  creating  an  animation.  In  addition,  dials 
were  coimected  to  the  model  using  function  networks,  so  that  the  user  could 
rotate. translate  and  scale  the  object  while  the  animation  was  progressing. 
The  number  of  frames  that  could  be  viewed  in  this  fashion  was  limited  by 
the  amount  of  memorv  avialable  on  the  PS  340  since  all  the  frames  had  to 
be  present  in  memory  at  once. In  this  case.  2  megabytes  (2  x  1024^)  were 
present,  allowing  no  more  than  15  frames  of  fiber  and  fluid  marker  data  to 
be  displayed.  The  implementation  of  the  dynamic  display  program  involved 
"hree  tasks:  downloading  the  display  structures;  animating  the  display;  and 
creating  the  user  interface.  These  are  discussed  below. 

6.1  Display  Structures 

As  noted  in  Section  4,  the  objects  to  be  displayed  were  the  fibers  and  the  fluid 
markers  (velocity  field  display  is  discussed  in  Section  7.3).  The  programs 
to  download  the  display  objects  made  heavy  use  of  the  PS  340  Graphic 
Subroutines  and  hierarchical  structuring  capabilities.  In  order  to  animate 
the  display,  each  frame  was  defined  to  be  a  separately  named  object  in  the 
PS  340.  Two  programs  were  written:  one  for  fibers  and  one  for  markers. 
These  were  handled  separately,  since  the  number  of  frames  to  be  loaded  is 
dependent  on  the  PS  340  memory,  it  was  possible  to  view  25  frames  of  fibers 
or  majkers,  but  only  15  frames  if  both  were  loaded  together. 

As  described  in  Section  4,  fibers  are  orgamzed  into  groups.  Using  the 
PS  340  VECTOR-LIST  command,  each  group  was  defined  as  a  separate 
display  structure  and  fibers  were  depicted  using  line  segments  joining  each 
point  to  its  next  neighbor.  The  last  point  vv  .s  joined  with  the  first.  Similarly, 
each  shell  of  fluid  markers  was  defined  as  a  separate  display  object,  but  were 
depicted  as  individual  points. 

6.2  Animating  the  Display 

The  next  task  was  to  construct  a  function  network  to  animate  the  display. 
Recall  in  Section  3.2  that  the  PS  300  command  language  provides  commands 
for  structural  attribute  setting,  which  includes  conditionally  displaying  a 
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part  of  a  model's  structure.  There  are  two  commands  that  modify  the  level 
of  detail.  The  command  SET  LEVEL.OF -DETAIL  THExN  <  obj  >  sets 
the  current  level  of  detail  to  display  for  the  display  structure  <  obj  >. 
The  IF  LEVEL-OF-DETAIL  <  condition  >  THEN  <  obj  >  will  display 

<  obj  >  if  <  condition  >  evaluates  to  true,  where  <  condition  >  is  either 

<  k.  <=  k  OT  =  k  where  k  is  some  integer.  This  allows  display  objects  to 
become  progressively  more  complex  by  increasing  the  level  of  detail,  but  it 
also  provides  a  mechanism  for  animating  the  display.  A  function  network 
to  increment  the  level  of  detail  at  regular  time  intervals  is  connected  to  the 

0    display  structure. 

6.3  User  Interface 

A  second  function  network  was  constructed  to  allow  user  interaction  with  the 
ilisplay.  This  function  network  cormected  the  dials  to  nodes  in  the  display 
structure  to  rotate,  translate  and  scale  the  object.  The  user  could  also  select 
the  number  of  frames  to  be  displayed  per  second.  The  function  keys  were 
used  to  start  and  stop  the  animation  and  to  reset  the  display  to  its  starting 
configuration.  Finally,  information  was  displayed  on  the  screen  indicating 
the  amount  of  rotation,  translating  and  scaling,  as  well  as  available  memory. 

6.4  Other  Developments 

6.4.1      Controllmg  the  Display  Hierarchy 

After  the  basic  features  of  the  dynamic  display  program  were  implemented, 
severed  issues  arose.  The  first  was  providing  the  user  with  a  method  for 
^electing  objects  in  the  hierarchy  to  display,  namely  groups  of  fibers  and 
shells  of  fluid  markers.  Commands  could  be  tvped  into  the  PS  340  directly, 
but  this  required  the  presence  of  a  specialist  who  knew  the  interal  organiza- 
tion of  the  program.  A  small  menu-driven  program  was  written  in  PASCAL 
where  the  user  specified  which  groups  and  shells  to  display.  This  was  far 
easier  to  implement  than  constructing  a  function  network,  since  the  VAX 
could  issue  commands  to  turn  on  and  off  display  components  easilv  using 
Graphic  Subroutines.  The  program  worked  uses  a  "toggling"'  approach:  the 
fijst  time  the  user  selects  a  group,  it  is  turned  off  and  the  second  time,  it  is 
turned  back  on.  This  is  illustrated  in  Section  5. 
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6.4.2      Stereo  Display 

Another  issue  was  better  three  dimensional  representation  of  the  display. 
The  PS  340  provides  prespective  projections  and  depth  cueing  to  clarify 
depth  relationships,  but  the  complexity  of  objects  being  displayed  demanded 
still  better  techniques.  To  this  end,  a  stereoptic  display  feature  was  devel- 
oped. 

The  theory  of  stereoptic  display  is  simple.  Compute  two  views  of  the 
object  to  be  displayed:  one  as  it  would  look  from  the  left  eye  and  one  from 
the  right  eye.  Then  present  these  to  the  viewer  so  that  the  left  eve  receives 
only  the  left  view  and  the  right  eye  receives  the  right  view.  The  computation 
of  the  different  views  was  done  usmg  an  algorithm  similar  to  that  given  in 
'51.  To  differentiate  the  displays  properly,  a  red  and  blue  color  filter  were 
obtained  that  matched  the  red  and  blue  colors  of  the  PS  340  display^  Thev 
had  the  property  that  the  red  filter  blocked  the  blue  light  and  vice  versa". 
Both  views  were  displayed  simultaneously;  the  right  eye  view  was  displayed 
in  red  and  the  left  eye  view  in  blue.  Placing  the  appropriate  filter  over  each 
eye  allowed  the  viewer  to  perceive  the  object  in  stereo. 


'The  filters  used  are  Kodak  Rattan  firers  #29  and  *47 

-Actually,  the  blue  filter  permitted  a  small  amount   of  red  li^ht  through.     This  was 
corrected  by  making  the  blue  image  brighter  than  the  red 
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7      Future  Development 

The  programs  described  above  have  all  been  implemented  and  are  in  use. 
However,  a  number  of  design  issues  are  pending: 

7.1  Use  of  the  PS  340  raster  display 

Figure  11  shows  the  results  of  early  experiments  with  the  solid  modeling 
capabilities  of  the  PS  340.  It  depicts  the  fiber  structure  as  a  smooth  surface 
made  up  of  ribbons.  This  is  a  realistic  depiction  of  the  fiber  structure  since 
the  heart  appears  to  the  eye  not  as  a  set  of  discrete  fibers,  but  as  a  smooth 
object.  Ribbons  were  used  to  allow  visualization  of  the  fiber  structure. 

Each  ribbon  is  formed  by  defining  a  sequence  of  triangles  whose  vertices 
are  pomts  on  adjacent  fibers.  Triangles  are  used  since  the  PS  340  requires 
the  solid  object  model  to  be  constructed  of  polygons  which  are  planar.  In 
order  to  allow  the  PS  340  to  render  these  ribbons  as  smooth  objects,  some 
approixmation  of  the  normals  to  the  surface  had  to  be  computed  for  each 
point  in  the  collection  of  polygons.  Treating  the  edges  of  the  triangles  as 
vectors,  the  normal  at  some  point  p  on  a  fiber  /  is  approximated  by  averaging 
the  cross-products  of  the  four  vectors  incident  to  p  on  the  ribbon  between 
fibers  /  and  /  +  1.  Thus,  each  point  p  on  f  has  two  normals  associated 
with  it:  the  normal  to  the  ribbon  defined  by  /  -  1.  /  and  the  ribbon  defined 
by  /i  /  +  !■  The  cross-product  average  is  then  divided  by  the  magnitude  to 
get  a  unit  normal.  These  methods  of  tiling  and  approximating  normals  are 
discussed  in  depth  in  :4j. 

The  raster  display  can  be  connected  directly  to  a  video  tape  controller. 
Figure  2  shows  the  connection  between  the  display  and  the  Lyon  Lamb 
VAS  IV  automated  video  tape  controDer.  A  set  of  subroutines  were  written 
to  control  the  video  tape  recorder  automatically  by  the  VAX  host. 

7.2  Video  tapes  of  the  vector  display 

We  would  stiU  like  to  prepare  video  tapes  of  the  wire-frame  models  used  on 
the  vector  display.  This  will  be  made  possible  using  the  writeback  feature 
released  in  version  A2.v01  of  the  PS  340  firmware.  This  allows  applications 
to  generate  lists  of  two  dimensioned  line  segments  that  are  being  drawn  on 
the  vector  display. 

A  progrcim  is  under  development  to  use  this  feature.  The  basic  idea  is  to 
display  one  frame  of  data;  signal  the  PS  340  to  "write  back"  the  line  segments 
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on  the  vector  display;  process  the  coordinates  and  generate  a  similar  drawing 
on  the  raster  display  using  Graphic  Subroutines  that  write  data  directly  to 
the  raster  b\iiFer.  This  would  allow  the  automated  video  tape  generation 
programs  of  Section  7.1  to  be  used.  Furthermore,  the  write-back  feature 
would  allow  us  to  generate  vector  drawings  on  any  output  device,  since  they 
can  be  converted  to  a  sequence  of  subroutine  calls  to  any  subroutine  package 
available  on  the  host  machine. 

7.3      Vector  Field  Display 

Another  area  of  research  is  devising  algorithms  for  displaying  a  three  dimen- 
sional vector  field,  such  as  the  velocity  field.  Current  literature  on  graphics 
provides  little  information  on  the  subject.  The  problem  is  to  display  the 
vectors  without  cluttering  up  the  screen.  Two  dimensional  vector  fields  do 
not  present  the  same  problem,  since  one  can  compute  a  stream  function. 
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8      Summary  and  Conclusions 

Development  of  the  graphics  tools  for  the  Evans  &  Sutherland  PS  340  system 
has  proved  successful.  It  has  reduced  development  time  of  HEART3D  by 
allowing  us  to  see  what  is  actually  happening  in  the  simulation.  A  number  of 
still  photographs  and  video  tapes  have  been  made  from  the  display,  enabling 
us  to  share  our  experiences  with  HEART3D  to  a  wider  audience.  Thus  it 
has  satisfied  the  two  major  requirements  of  its  design. 

The  PS  340  graphics  system  has  proven  to  be  very  useful  for  this  ap- 
plication. Its  interactive  capabilities  allowed  the  user  to  manipulate  the 
display  in  a  natural  and  enjoyable  way.  Its  command  language  was  powerful 
enough  to  allow  us  to  concentrate  on  the  geometrical  aspects  of  HEART3D 
data  with  httle  concern  for  issues  such  as  hardware  configuration,  internal 
representation  of  graphical  objects  and  so  on.  However,  the  PS  340  is  not 
without  its  disadvantages: 

•  Poor  interface  capabilities.  The  existing  communication  links  between 
the  PS  340  and  the  VAX  host  are  extremely  slow.  Our  application  re- 
qmres  the  graphics  display  to  access  a  very  large  data  base.  Currently, 
we  are  limited  to  viewing  only  that  which  can  be  placed  in  the  PS  340 
memory.  A  high-speed  communications  line  would  allow  the  PS  340 
to  access  the  data  base  on  the  host  machine. 

•  Inadequate  development  tools.  Function  networks  are  quite  difficult  to 
prograjn  by  hand.  Development  tools  are  necessary  to  aid  the  pro- 
grammer in  designing,  implementing  and  debugging  these  networks 
and  integrating  them  into  the  application.  Existing  tools  provided  by 
Evans  &  Sutherland  are  sadly  lacking  in  this  area. 

There  are  many  problems  still  to  be  resolved.  Some,  such  as  solid  object 
modeling  of  the  fiber  structure,  have  been  carefully  considered  and  simply 
await  implementation.  Others,  such  as  use  of  the  "write-back"  capabihties. 
require  enchcuunents  to  the  hardware  of  the  PS  340  before  they  can  be 
developed.  Still  other  issues,  such  as  display  of  the  velocity  vector  field,  are 
in  their  infancy;  manv  ideas  must  be  tested  before  a  solution  is  forseeable. 
It  is  certain,  however,  that  graphics  programis  in  general  and  the  PS  340 
graphics  system  in  particular,  will  continue  to  play  an  important  role  in  the 
presentation  of  results  from  the  HEART3D  program. 
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A     User  Manual 

This  appendix  is  a  brief  guide  for  users  of  the  three-dimensional  graphics 
programs  for  display  of  HEART3D  results.  It  describes  the  implementation 
at  NYU  on  the  ACF  VAX/ VMS  cluster  and  assumes  some  knowledge  of  the 
VMS  operating  system. 

A.l      Setting  up  data  files 

The  first  task  is  transfer  the  ASCII  data  files  generated  by  HEART3D  to 
the  graphics  database,  convert  them  to  binary  format  and  create  a  list  file 
containing  the  names  of  the  data  files  to  be  downloaded  to  the  PS  340.  It 
is  assumed  that  data  files  have  the  following  name  form.ats: 

•  Fiber  data  files:     TuuXvvKwwww.; 

•  MarkeT  data  files:     TuuXvvMKKwwww .  ; 

where  uu  is  the  dimensions  of  the  experiment,  vv  is  the  current  experiment 
nimiber  and  wvww  is  the  KLOK  value  of  the  data  file. 

1.  Log  in  to  the  account  containing  the  graphics  database. 

2.  Make  sure  that  there  is  enough  space  to  hold  the  data  files  in  question. 
ASCII  data  files  may  be  placed  in  any  sufficiently  large  directory  area 
but  the  binary  files  wiU  be  placed  in  the  DATA  directory.  Each  binary 
file  is  approximately  33%  of  the  size  of  the  corresponding  ASCII  file. 

3.  Transfer  the  ASCII  files  to  a  large  directory  space.  The  method  of 
transfer  depends  on  when  the  ASCII  files  are  located.  If  they  are  in 
another  machine,  use  the  FTP  program.  If  they  are  on  magnetic  tape, 
some  special  handhng  may  be  necessary. 

4.  Set  the  default  directory  to  be  that  which  contains  the  ASCII  data 
files. 

5.  Type  cvtf ib2  <  fname  >  to  convert  the  files  containing  fiber  data. 
The  symbol  <  fname  >  is  the  first  six  characters  of  the  fiber  data  file, 
namely  TuuXvv  where  uu  and  vv  are  defined  above.  This  program  will 
look  through  the  current  directorv,  find  all  the  fiber  data  files,  convert 
them  to  binary  and  place  the  converted  file  in  the  D.A.TA  directory. 
The  ASCII  files  are  not  modified. 


6.  Type  cvtmrk2  <  /name  >  to  convert  the  marker  data  files.  The  sym- 
bol <  fname  >  is  defined  as  above.  This  program  works  in  the  same 
fashion  as  cvtf  ib2. 

7.  Create  a  list  file  using  a  text  editor.  It  is  useful  to  capture  a  listing  of 
the  DATA  directory  into  a  file  and  then  edit  it  to  list  only  those  files 
to  be  viewed.  The  dvnamic  data  program  has  a  limitation  of  10  files. 
Be  sure  that  the  list  fiJe  contains  the  directory  name  before  each  file 
name  and  that  the  name  is  not  followed  bv  a  period  or  the  extension. 

You  are  now  ready  to  execute  the  hoartuser  program  to  download  the  files 
to  the  PS  340. 

A. 2      Using  the  dynamic  display  program 

The  menu-driven  program  that  wiU  (eventually)  administrate  all  commu- 
nications between  the  host  and  the  PS  340  is  named  heartuser.  Only 
the  dynamic  display  program  is  currently  available,  although  the  menu  lists 
other  choices. 

To  use  heartuser.  one  must  first  log  in  to  the  account  containing  the 
graphics  database  from  the  PS  340.  To  do  this,  the  PS  340  must  be  in 
terminal  emulation  mode.  This  is  established  by  pressing  the  LINE-LOCAL 
key  on  the  PS  340  keyboard.  The  PS  340  emulates  a  VTIOO  terminal,  so  be 
sure  to  set  the  terminal  properly  after  logging  in. 

Next,  follow  these  steps: 

1.  Type  set  def  hgs$heart3d_ps300.This  will  place  you  in  the  direc- 
tory containing  heartuser. 

2.  Type  flheartuser.  This  will  display  the  initial  menu. 

3.  Type  1  to  select  the  dynamic  display  program.  Be  sure  that  this  is 
followed  by  pressing  the  ENTER  key  (as  with  all  other  commands). 
The  dynamic  display  menu  will  be  shown. 

At  this  point,  you  have  several  choices.  Pressing  the  key  corresponding  to 
an  entry  on  the  screen  wiU  download  the  function  networks  and  then  ask 
you  for  the  name  of  the  list  file.  Use  the  file  generated  in  Section  A.l.  Be 
sure  to  provide  the  directory  wher  the  list  file  resides.  Then  the  data  will 
be  downloaded  and  ready  for  use. 
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A.3      Controlling  the  PS  340  display 

Let  us  suppose  that  you  have  successfully  downloaded  both  fiber  and  marker 
data.  At  this  point,  the  initial  configuration  will  be  displayed  (that  is, 
data  from  the  first  data  file  in  your  list  file).  The  dials  and  a  number 
of  function  keys  are  labeled.  The  dials  labeled  TRANS  X, TRANS  Y, TRANS 
Z  will  translate  the  image  in  the  x.y,  and  z  directions,  respectively.  The 
dials  labeled  ROT  X.ROT  Y.ROT  Z  rotate  the  image  about  the  x,y  and  z  axes 
respectively.  SCALE  will  change  the  size  of  the  object  and  FR/SEC  changes 
the  speed  of  the  animation. 

To  start  the  animation,  first  hold  down  the  SHIFT  kev  and  press  the 
LINE-LOCAL  key  at  the  same  time.  Then  press  the  function  key  labeled 
START.  The  animation  will  loop  through  all  frames  downloaded.  To  change 
the  speed,  turn  the  FR/SEC  dial.  The  informational  display  above  the  object 
wiU  reflect  the  current  number  of  frames  per  second  being  shown.  Note  that 
the  function  key  labeled  START  is  now  labeled  STOP.  Pressing  this  key  wiU 
freeze  the  display  at  the  current  frame.  All  function  keys  (except  the  RESET 
key)  work  in  this  fashion.  For  example,  pressing  the  FIB  OFF  key  wiU  turn 
off  the  fiber  display,  pressing  the  MARK  OFF  key  will  turn  off  the  marker 
display,  etc. 
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